Evaluation of second-level inference in fMRI analysis

We investigate the impact of decisions in the second-level (i.e., over subjects) inferential process in functional magnetic resonance imaging on (1) the balance between false positives and false negatives and on (2) the data-analytical stability, both proxies for the reproducibility of results. Second-level analysis based on a mass univariate approach typically consists of 3 phases. First, one proceeds via a general linear model for a test image that consists of pooled information from different subjects. We evaluate models that take into account first-level (within-subjects) variability and models that do not take into account this variability. Second, one proceeds via inference based on parametrical assumptions or via permutation-based inference. Third, we evaluate 3 commonly used procedures to address the multiple testing problem: familywise error rate correction, False Discovery Rate (FDR) correction, and a two-step procedure with minimal cluster size. Based on a simulation study and real data we find that the two-step procedure with minimal cluster size results in most stable results, followed by the familywise error rate correction. The FDR results in most variable results, for both permutation-based inference and parametrical inference. Modeling the subject-specific variability yields a better balance between false positives and false negatives when using parametric inference

Evaluating of bootstrap procedures for fMRI data

Over the last decade the bootstrap procedure is gaining popularity in the statistical analysis of neuroimaging data. This powerful procedure can be used for example in the non-parametric analysis of neuro-imaging data. As fMRI data are complexly structured with both temporal and spatial dependencies, such bootstrap procedures may indeed offer an elegant solution. However, a thorough investigation on the most appropriate bootstrapping procedure for fMRI data has to our knowledge never been performed. Friman and Westin (2005) showed that a bootstrap procedure based on pre-whitening the temporal structure of fMRI time series is superior to procedures based on wavelets or Fourier decomposition of the signal, especially in the case of blocked fMRI designs. For time-series, several bootstrap schemes can be exploited though (see e.g. Lahiri, 2003). For the re-sampling of residuals from general linear models fitted on fMRI data, we examine more specifically here the differences between 1) bootstrapping pre-whitened residuals which are based on parametric assumptions of the temporal structure, 2) a blocked bootstrapping which avoids making such assumptions (with several variants like the circular bootstrap,. . . ), and 3) a combination of both bootstrap procedures. We furthermore explore whether the bootstrap procedures is best applied before or after smoothing the volume of interest. Based on real data and simulation studies, we discuss the temporal and spatial properties of the bootstrapped volumes for all possible combinations and nd interesting differences

Data-analytical stability in second-level fMRI inference

We investigate the impact of decisions in the second-level (i.e. over subjects) inferential process in functional Magnetic Resonance Imaging (fMRI) on 1) the balance between false positives and false negatives and on 2) the data-analytical stability (Qiu et al., 2006; Roels et al., 2015), both proxies for the reproducibility of results. Second-level analysis based on a mass univariate approach typically consists of 3 phases. First, one proceeds via a general linear model for a test image that consists of pooled information from different subjects (Beckmann et al., 2003). We evaluate models that take into account first-level (within-subjects) variability and models that do not take into account this variability. Second, one proceeds via permutation-based inference or via inference based on parametrical assumptions (Holmes et al., 1996). Third, we evaluate 3 commonly used procedures to address the multiple testing problem: family-wise error rate correction, false discovery rate correction and a two-step procedure with minimal cluster size (Lieberman and Cunningham, 2009; Bennett et al., 2009). Based on a simulation study and on real data we find that the two-step procedure with minimal cluster-size results in most stable results, followed by the family- wise error rate correction. The false discovery rate results in most variable results, both for permutation-based inference and parametrical inference. Modeling the subject-specific variability yields a better balance between false positives and false negatives when using parametric inference

Spontaneous variability of pre-dialysis concentrations of uremic toxins over time in stable hemodialysis patients

Background and aim : Numerous outcome studies and interventional trials in hemodialysis (HD) patients are based on uremic toxin concentrations determined at one single or a limited number of time points. The reliability of these studies however entirely depends on how representative these cross-sectional concentrations are. We therefore investigated the variability of predialysis concentrations of uremic toxins over time. Methods : Prospectively collected predialysis serum samples of the midweek session of week 0, 1, 2, 3, 4, 8, 12, and 16 were analyzed for a panel of uremic toxins in stable chronic HD patients (N = 18) while maintaining dialyzer type and dialysis mode during the study period. Results : Concentrations of the analyzed uremic toxins varied substantially between individuals, but also within stable HD patients (intra-patient variability). For urea, creatinine, beta-2-micro-globulin, and some protein-bound uremic toxins, Intra-class Correlation Coefficient (ICC) was higher than 0.7. However, for phosphorus, uric acid, symmetric and asymmetric dimethylarginine, and the protein-bound toxins hippuric acid and indoxyl sulfate, ICC values were below 0.7, implying a concentration variability within the individual patient even exceeding 65% of the observed inter-patient variability. Conclusion : Intra-patient variability may affect the interpretation of the association between a single concentration of certain uremic toxins and outcomes. When performing future outcome and interventional studies with uremic toxins other than described here, one should quantify their intra-patient variability and take into account that for solutes with a large intra-patient variability associations could be missed

The empirical replicability of task-based fMRI as a function of sample size

Replicating results (i.e. obtaining consistent results using a new independent dataset) is an essential part of good science. As replicability has consequences for theories derived from empirical studies, it is of utmost importance to better understand the underlying mechanisms influencing it. A popular tool for non-invasive neuroimaging studies is functional magnetic resonance imaging (fMRI). While the effect of underpowered studies is well documented, the empirical assessment of the interplay between sample size and replicability of results for task-based fMRI studies remains limited. In this work, we extend existing work on this assessment in two ways. Firstly, we use a large database of 1400 subjects performing four types of tasks from the IMAGEN project to subsample a series of independent samples of increasing size. Secondly, replicability is evaluated using a multi-dimensional framework consisting of 3 different measures: (un)conditional test-retest reliability, coherence and stability. We demonstrate not only a positive effect of sample size, but also a trade-off between spatial resolution and replicability. When replicability is assessed voxelwise or when observing small areas of activation, a larger sample size than typically used in fMRI is required to replicate results. On the other hand, when focussing on clusters of voxels, we observe a higher replicability. In addition, we observe variability in the size of clusters of activation between experimental paradigms or contrasts of parameter estimates within these